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OPPOSITE-FORM INVENTORIES, (2) TO CROSS VALIDATE 
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Problem 

The present study had three general purposes: first, to 

investigate the reliabilities of opposite-form inventories, second, 
to cross validate the opposite-form inventories, and third, to 
study the pattering of the external validity of the opposite -form 
.approach. 

Objectives 

1. To compare the internal consistency reliabilities of Opposite 

Forms A and B: a) the odd-even correlations, and b) the 

Kuder-Richardeon correlations . 

2. To determine the test-retest reliabilities of Poi^ A and B. 

3. To cross validate the test-retest reliabilities of Forms A 
and B, using two sub-groups. 

4. To cross validate or to compare the opposite-form reliabilities 
obtained in the present study with those of an earlier study. 

5. To determine the validity coefficients of the opposite-form 
inventories with an educational criterion. 

6. To comj^re the correlated r*s between Forms A vs. B, A' vs. B' 
all of these forms correlated with an education criterion. 

Related Research 

The development of Opposite forms may be seen from the following. 

In 1923^ Cady made a selection of questions from the Woodwoirbh material, 
with considerable modification of phrasing. In order to give the questions 
an additional effectiveness, another form was prepared in which every 
question in the first form was reversed in the other. Cady noted a lacjs: 



2 



of clearness and directness in many questions. He admitted that it was 
very difficult to change a statement from the natural direct way of 
ejcpression into a question that said the same thing yet required an 
opposite reply. The result was a number of expressions which were 
practically double negatives. At any rate, the various reliabilities 
of his questionnaire ranged form .25 to .42. He concluded that it is 
open to question whether a single test containing items in direct and 
reversed forms would be as valuable as the same number of carefully 
selected questions divided into two forms and given at different periods. 

In 1925 Laird described the Colgate Mental Hygiene Test, and Hoitsraa 
(1925) gave some statistical results from its use. This inventory was 
an extension of. the Woodworth (1918) material with fifty additional 
items dealing with introversion and extroversion. Here the main interest 
centers on the introversion and extroversion additions, since' these 
parts were dealing -with some kind of opposite forms of items. At any 
rate, Hoitsma did report the r’s of ».22 to -.45 between introveitj and 
extrovert scores. At about the same time Symnnds ( 1925 ) constructed a 
social attitudes questionnaire to measure liberalism. In order to test 
the reliability of the questionnaire, a second form T4as prepared containing 
the same questions so expressed that the answers would be the opposites 
to the answers appropriate for the items in the original questionnaire. 

The correlation of the two forms was .67. After 1925 the practice of 
constructing reversed forms was not en 5 >loyed until the appearance of Adorno’s 
book on The Authoritarian Personality in 1950 • The main concern here 



was the P~Scale in The Authoritarian Personality * A group of writers 



attenrpted to constru.et a reversed form of the F-Scale to suit the 
purposes of their own investigation. Thus once again the practice of 
constructing reversed forms was in vogue. 

Among this group of liters. Chapman and Camphell (1957) employed 
two sets of ten and sixteen items in their original form, arid about the 
same number in the reversed form. The correlations between the scores on 
original and reversed items- on two occasions were only .17 and -.01. 

Chapman and Campbell concluded that the achievement of a successful 
reversal would be a very difficult task. Other writers also had ideas of 
reversed form construction for this inventory. Jackson and Messick (1957) 
constructed the reversed P~Scale by rewriting each item so that the content 
would appear to reflect a viewpoint opposed to the original, while retaining 
a similar style of expression. Later, Jackson, Messick, and Solley (1957) 
reported a correlation of .35 between their original and reversed forms. 

Bass (1955) .constructed the G-scale on the' basis that each new statement 
was opposite in meaning-'«or as opposite as the author could make it— to an 
original statement in the G-scale« Bass foujid that the r between P- and G- 
scales was ».20« Leavitt, Hax, and Roche (.1955) selected half of the 
P-scale items and reworded them into negative forms so that authoritarianism 
would now be indicated by disagreement rather than agreement with the items, 
and found that the r’s between the reversed and uncever-sed halves with five 
small samples were between -ofe2 and .^1-5, except one r of -.66 with an N of 6. 



4 



These studies on the low r’s hetween the original and the reversed 
P-scales were also cited and discussed hy Christie^ Havel and Seidenberg 
(1958), who attempted to construct a reversed P-scale based on the criteria 
of logical and psychological opposition, and the^ avoidance of extremity 
of wordingo The correlations between the scores from the fifteen original 
and the fifteen reversed items were between ,lk and . 45 . At about che 
same time Spivack (1956) developed an instrument to measure certain aspects 
of self -acceptance and self -rejection. Upon examination of Spivack’ s 
inventory, it was interesting to discover that the inventory contained 
both a positive' and a negative form— a set of opposite forms. Althou^ 
this inventory consisted of the items to measure self -acceptance and self- 
re jecticn, Spivack apparently had not touched upon the idea of opposite 
forms since nowhere was this idea mentioned, nor were correlations 

I 

given between the self -acceptance and self -rejection items. 

In general, investigators, with the exception of Symonds, have re- 
ported a low relationship or an absence of relationship between the original 
and the reversed foims as constructed by each investigator according to 
his particular concept. At this point the following questions seem peitinent: 
Why was there such a low relations’hip, or an absence of relationship, as 
found by these researchers? Why was Symonds’ finding an exception? Very 
recently, Ong (1965), in his book called, The Opposite -Form Procedure in 
Inventory Construction and Research , has atten^ted to explain the rationale 
behind the low relationship between the original and reversed forms found 
by these investigators and the reason behind Symonds’ apparent success in 
obtaining a moderate relationship. Ong also has investigated certain aspects 
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of the effectiveness and flexlhility of oppositely-stated items for use in 
construcing personality inventories. He has refined and developed certain 
concepts of oppositeness discussed some related research on some aspects 
of the concepts of opposite forms, analyzed the items of opposite forms 
of an inventory, illustrated the practical construction of the opposite 
forms of seven inventories, and carried out a series of experimental studies 
using the constructed opposite forms of these inventories. A general 
conclusion vas that opposite forms of the Inventory type containing items 

with opposite directional idea could he constructed with moderately high 

} 

reliahilltyo 

There has been a fashipn for the modem writers to construct opposite 
forms not for the main purpose of research in opposite form construction, 
hut for the purpose of investigating an area called "acquiescence" or 
"response style" in personality inventories. Rorer (1963) has r^eviewed' 

W- •* • 

at length studies in this area. It seems tha,t the Inventory used most fre- 
quently in the constmctlon of such forms is the MMPI, or its derivatives. 

The most recent exair^les are those of Rorer and Goldherg ( 19^5 ; '^ho 

constructed the reversed MMPI items in an attempt to argue for the non- 
existence of acquiescence. It seems that their correlations ml^t have 
heen influenced hy the fact that they found items in their reversed J®iPI 
which were quite inei’feetive to. attain reversal. Also they have not yet 

' performed the retest on their reversed MMPI . 

Earlier, Ong (196I?) has demonstrated cerbain effective procedures in 
opposite or reversed form construction. But there are cerbain special areas 
remained to he investigated, for example, the retest using the opposite forms, 
the comparison of the internal consistency reliahility between the opposite 
forms, the cross validation of the opposite fozms, and the correlation or 
prediction of an educational criterion using opposite forms. 



Proeediire 



The opposite “'forms A and B of five inventories constructed hy Ong 
(l96p)-*Test Ip Carter's (1958) Mechanics of Study Procedure (good items); 
Test II., Allport's ( 1928 ) Ascendant-Submissive Behavior; Test III, Gough's 
( 1953 ) Honor Point Scale; Test IV, Edward's (1957) Social Desirability 
Scale; and Test V, Spivack’s (1956) Self -acceptance and Self -rejection 
(good items)— were given to a general psychology class and three other 
classes in development, learning and evaluation. Subjects consisted of 
96 mal 6 s and 95 females, a total of I 9 I freshmen, sophomores, and juniors 
enrolled in the author's classes in one quarter. The five inventories 
were given in four sittings, each about two days apart, that is, test 
Form A, test Form B, retest Form A’ , and retest Form B. The order of 
giving these four forms was -as follows: 



Class 


ToH. 


I 

BoHo 


ToH. 


n 

B.H. 


III 

ToH. B.Ho 


IV 

T.H. B.H. 


1st round 


A 


B 


A 


B 


A 


B 


A 


B 


2nd round 


■ B 


A 


A' 


•ym A 


B 


A 


A’ 


B' 


3rd round 


A’ 


B’ 


B 


A 


A' 


B' 


B 


A 


4th roimd 


B' 


A’ 


B? 


A’ 


B' 


A' 


B' 


A' 



For exaa^-le, in the first round, the top half of the students in each of 
four classes (top ha'lf vs. bottom lialf alphabetically) took Foim A first 
while the bottom half took Form B first; in the second round, the procedure 
vas reversed for two classes of the four, that is, the top half took Form 
B, while the bottom half took Form. A. The remining two classes took the 
retest; that is, top half took Form A', while bottom half took Form B', etc. 
The inventories were scored as discrepancies with the key. ‘.rhe course grade 
for each student was calculated in terms of T-scores, using the average 
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T-scoreB of atout ten weelsly examinations. The odd-even r’s and the 
KJader-Eiehardson r’s from the earlier study (Ong^, 19^5) were used to 
calculate the significance of the correlated r*s "between the opposite- 
forms A and E. The various reliability coefficients of the Forms A 
and B;, A” and B’ of the five inventories were calculated for the 
present saarple. Also the correlations between the Forms A and A’ 
and E*j, all with course grade were calculated. Then various comparisons 
were made for different correlations. 

Analyses of the Data and Findings 

Table I presents con 5 )arison of Ong*s (19^5 ) odd-even correlated 
r“s between the Opposite Forms A and B. Out of the five tests there 
exists a significant difference in only one test, that is, Test V. 

Table 2 again gives comparisons of Ong’s Kuder^Richardson correlated r’s 
bet-ween the Opposite Forms A,and B. None of the differences is signi« 
fieant. Consequently, with one significant, difference out of ten 
comparisons in internal consitency reliabilities between Opposite Forms 
A and E.; one may ha^'/e some moderateily hl^x degree of confidence that 
these twof^rms are comparable. 

Table 3 lists the various reliability coefilcierits of the Opposite 
Forms of the five inventories of the present sanp>le. All the r’s are 
significant and way above .01 level. In general, the test-retest r’s 
seem to be a little higher than the opposite-form r’s. These results 
conform to the usual observation that test-retest r’s are higher than 
the parallel-fom r’s. Table ^ presents the cross validation or com- 
parison of the test-retest re.liabilities of Forms A and B, using the two 
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largest sub-groups (freshmen vs. ju 2 iiors). None of these differences 
between the test-retest r’s, which were converted to z’s^ was signi- 

T 

f leant. Thus these results show that these Opposite -Poniia are quite 
co3roa2able from one group to the bther^ as far as the college popu- 
lation is concerned o 

Table 5 lists the opposite-foim r‘s of Ong’s earlier and the 
present investigation. Since the retest was not done in the earlier 
•study^ the retest r’s are. listed only for the present study. Also the 
r’s prc'iected tc 100 items are listed. Table 6 presents a cross mlida- 
tion or con^jarison of the opposite-form reliabilities obtained in the 
present college saurple with those of an earlier adult-extension san^le. 
Again none of these differences is significant. Thus these results show 
that- the Opposite -Forms are q;uite comparable, at least from undergraduate 
college sample to the adult-extension saa^le. 



Table 7 lists the validity coefficients of the opposite-form inven- 



tories -with course grade. The r^s of four out of five tests are signifi- 
cant at least at the ,05 level. The minus r'S conform with the intent 
of the test eonst:cuctor since the inventories were scored as discrepancies 
with the key. The results show that Tests I,. Ill, IV and V correlated or 
predicted 'the criterion, co'or'se grade, about equa.lly well, and at least 
better than Test II, Yet, the correlation or prediction is not too much 



above significance, or only several percent better t7asn. dhance. Table 8 
presents the comparisons of the correlated r’s- between Opposite Forms 

UM 

A vs, B, A' vs, all of these forms correfiated with course grade. Since 
in Test II, only one r, that is, ,l6, is barely significant out of four. 
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con^Jariscn is not made for this test. Again, none of these differences 
in the remaining four tests is significant. Consequently, the opposite- 
forms are comparable, as far as the validity coefficients using an out- 
side educational criterion is concerned. .The implication of these results 
seems to he that the correlation ori prediction of an educational criterion 
.course grade, is just as effective in using the opposite -forms of an 
inventory as is the original. 

I 

Conclusions and Implications 

Within the limits of these samples, the following conclusions and 
in^lieations seem warranted: 

1. With one significant difference out of ten con5)arisc:n.s in internal 
consistency reliabilities between the opposite -forms, the eompara- 
bility of these forms* was considered to be moderately high. 

2. All the various reliability coefficients, including the tsst-retest 
r's, yrere hi^ly significant. 

3 . There was no si^ificant difference in the cross validation or com- 
parison of test-retest reliabilities of the opposite forms, using 

a freshznan and. junior sanple. 

4i There was no signi.fiea*Tit difference in the cross iralidation or com- 
parison of opposite-form reliabilities obtained in the present college 
sample with those of an earlier adult-extension san^le . 

5 . The validity coefficients of the opposite-form inventories with an 
educational criterion, course grade, were s'ignificant for four out 
of five inventories. These four tests correlated or predicted course 
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grade about equally well,.. but only about several percent better 
than chance . 

6 . .There 'Was no significant diff erehce ' between the correlated r‘s 
of the opposite fonns correlated with course grade. 

7. As a general conclusion, these opposite forms using several meabures 
of cong?arieon were hi^ly comparable. 

8 . The in5)lication of these results seems to be that the correlation 
or prediction of an educational criterion, course grade, is just 
as effective in using the opposite -forms of an inventory as is the 
original. 



er|c 
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TABLE 1 



) 

Ccffl^riflon of Ong's Odd -Even Correlated r’s 

I 

iDetween Opposite Forms A and B 
( N = 6l adu3.t extension students) 



Test 


Form 


Odd-Even r 


r^ 




^AB 


^AB 


I 


A • 


v 04 


.0016 


• 




• , 






• 




.06 


. .67 


.4489 




B 


.10 


.01 








II 


A 


* * * 

.60 


:36 














.06 


.83 


.6889 




B 


.66 


.4356 








III 


A 


.49 


.2401 
















.06 


.72 


.5184 




‘ B . 


.43 


.1849 








IV 


A 


.42 


.1764 


.09 


.77 


.5929 




B 


.51 


■ .2601 








V 


A 


.52' 


.2704 


.20' 


.67 


.4489 




B ■ 


.72 ■ 


.5184 
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t 

.57 

1 ^ 05 * 

.672 

1.18 

2.70 
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TABLE 2 

Comparison of Ong’s Kuder-Richardson Correlated 
r*s "between Opposite Forms A and B 
(N = 6l Adult extension students) 



Test 


Form 


K-R . 

.•y 


r2 


I 


A 


.19 


.0361 




B 


.23 


.0529 


II 


A 


:73 






' B 


.71 


.5041 


III 


A 


.41 


. .1681 




B 


.29 


.0841 


IV 


A 


.59 


.3481 




B 


.63 


.3969 


V 


A 


.72 


.5184 




B 


.65 


.4225 




.04 



.02 



.12 



.04 



.07 



^AB“ 

.67 




.72 



.77 



.67 



.4469 



.6889 



.5184 



.5929 



.4489 



o 
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.388 



.398 



1.344 



.592 



1.001 
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TABLE 3 

The Various Re 3 . 1 abllity Coejpficients of 

» • 

Opposite -Forms of Five Inventories 
(Total N s 191) 



Forms 


I 


AA* 


.75 


BB* 


.57 


AB 


*53 


A'B* 


.70 


AB’ 


.66 


BA’ 


.61 



. ' 

Test 

II III 

.85 .80 

.89 .70 

.75 .73 

.83 .68 

.79 .69 

.72 



IV V 

. 81 ^ . 81 ^ 

.81 .76 

.7^ .72 

.78 .73 

.73 .68 

.76 



79 



73 



l6 



TABLE k 

4 

Comparison of Test-retest r's between two Sub-groups. 



Test 


• 


Forms 


r 


z 




t 


P 


I 


Freshman N =75 


AA' 




.95 


.08 


.43 


N 




Junior N =52 




.70 


.87 










Freshman 


BB‘ 


,6k 


.76 




0 


N 




Jtinior 




.6h 


.76 








II 


Freshman 


AA* 


.82 


1.16 


.14 


f 76 


N 




Junior 


' / 


.77 


1.02 










Freshman 


BB* 


. .86 


1.29 


. .13 


.70 


N 




Junior 




.89 


I.I42 








III 


Freshman 


AA* 


.86 

.76 


1.29 


.29 


1.57 


N 




Junior 




1.00 










Freshman 


BB* 


.71+ 




.10 


.54 


N 




Junior 




.78 


1.05 








IV 


Freshman 


AA* 


.78 


1.05 


.02 


.11 


N 




Jimior 




.79 


1.07 










Freshman 




.80 


1.10 












BB* 






.03 


.16 


N 




Junior 




.^1 


1.13 








V 


Freshman 


AA* 


.83 

.76 


1.19 


.19 


1.03 


■ N 




Junior 




1.00 










Freshman 


BB’ 


.75 


.97 


.24 


. 1.30 


N 




Junior 




.62 


.73 









^ d, - .185 






miE 5 

Listing of the Opposite-Form r’s of Ong's 
Earlier and Present Investigations 



Earlier Investigation 
N = 6l Adult university 
Extension students 

Test 

I 

II 

III 

IV 

V 

Present Investigation 
N a 191 Freshmen, Sophomores, ’ 
and Juniors . i . 

■ II 

III 
IV 



r r projected to 

~100 items 



AB 


A'B' 


AB 


A'B’ 


.67 




.91 




.83 




.96 




.72 




.93 




.77 








.67 




.91 





■.53 


.70 


.85 


CVJ 

o\ 

• 


.75 


.83 


.94 • 


.96 


.73 


.68 


.93 




. 7 ^ 


.78 


.93 


\o 

vn 



V 



.72 



.73 



.93 



.93 



V 
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TABLE 6 



J 

Comparison of Opposite Form r’s "between 
Ong’s Earlier and Present Groups 



Test 


Opposite -Form 


r 


z 


Dz 


t 


p 


I 


Earlier N=6l 


AB 


.67 


.81 


.22 


1.47 


N 




Present N«191 


AB 


.53 


•59 












A‘B‘ 


.70 


.87 


.06 

% 


. »ko 


N 


II 


Earlier 




.83 


■ 1.19 


r 






- 


Present 


AB 


;75 


.97 


.22 


1.14-7 


N 






A‘B‘ 


.83 


1-1?. 


0 


0 


N 


Ill 


Earlier 


AB 


.72 


.91 


.02 


-13 


N 




Present 


AB 


.73 


.93 ■ 












A‘B‘ 


.68 


.83 


.08 


.53 


N 


IV 


Earlier 


AB 


.77 

.7k 


1.02 


.07 


A7 


N 




Present 


AB 


.95 










- 


■ A‘B' 


.78 


1.05 


.03 


■ .20 


N 


V 


Earlier 


AB 


.67 


.81 


110 


.67 


N 




Present 


AB 


.72 


.91 












A’B‘ 


.73 


.93 


.12 


.80 


N 



dg s .15 




liMHMiMtfilliMHr* 
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TABLE 7 

Validity Coefficients of the OppositerrForm 

I 

Inventories with Course Grade 

(N= 191) 

'test 



Form 


I 


II 


III 


IV 


V 


A 


-.2k* 


.16* 

1 . 


-.25* 


-.22* 


-.21* 


A» 


' -.2k* 


.11 


-.2k* 


-.21* 


-.25* 


B 


-.19* 


-.02 


-.21* 


-.17* 


-.28* 


B’ 


-.18* 


.06 


-.22* 


-.18* 


-.16* 



Minimum sig. r*s: with N = 191 

.15 at .05 
.19 at .01 

The inventories were scored as discrepancies with the key 

r 



TABLE 8 



Comparison of Correlated r's between Opposite -Pom 
A vs. B, A’ vs. B’, All of these Poms Correlated 
~ "" “with Course Grades (N«19l) 



Test 


Pom 


r 


r^ 




r I 


r V 


t 


P 










T 


AB or ' 


[ AB orl 
















A'B’ 


\ A'B'/ 






I 


A 


•‘,2k 


.0576 


.05 


.53 


.2809 


.711 


N 




B 


-.19 


.0361 














A' 


-.24 


.0576 


.06 


.70 


.4900 


1.06 


N 




B' 


-.18 


.0324 












II 


A 


.16 
















B 


-.02 
















A' 


.11 
















B' 


.06 














III 


A 


-.25 


.0625 


,o4 


.73 


.5329 


..751 


N 




B 


-.21 


.o44l 








1 






A' 


-.24 


.0576 , 


.02 


.68 


.li624 


.345 


N 




B' 


-.22 


.0484 












.:iv 


A 


-.22 


.0484 


.05 


.74 


.5476 


. .948 


N 




B 


-.17 


.0289 














A' 


-.21 


.o44l 


.03 


.78 


.6084 


.617 


N 




B' 


-.18 


.0324 












V 


■A 


-.'21 


.0441 


.07 


.72 


.5184 


1.30 


N 




B 


-.28 


.0784 












■ 


A' 


-.25 


.0625 


.09 


.73 


.5329 


1.69 


K 




B' 


-.16 


.0256 













* Since only one r, that is, .l6, is barely significant and the remaining 
three r's not significant, comgparison is not made for this test. 



